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This article reports on a new approach to properly analyze time series of dynamical systems which 
are spoilt by the simultaneous presence of dynamical noise and measurement noise. It is shown that 
even strong external measurement noise as well as dynamical noise which is an intrinsic part of 
the dynamical process can be quantified correctly, solely on the basis of measured times series and 
proper data analysis. Finally real world data sets are presented pointing out the relevance of the 
new approach. 
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A major challenge in analyzing time series originating 
from complex systems is to reveal the underlying process 
dynamics. Typically the simultaneous involvements of 
non-linearities, dynamical noise and measurement noise 
cause problems for many experimental situations and ac- 
count for the complexity of this task. The handling of 
these complications is the central concern of this paper. 

To extract an underlying signal disturbed by noise, lin- 
ear and non-linear predictor models or noise reduction 
schemes are widely used (for discussion see Q and refer- 
ences therein). Here we choose an alternative approach 
based on the broad class of Langevin processes which 
describes a variety of complex dynamical systems. 

Let us consider a one-dimensional Langevin process 
(the extension to more dimensions is straightforward) 
that is given by: 



d {1 \x) + sJd^Hx) r(t) 



(i) 



The term T(t) represents Gaussian white noise with 
(r(i)) = and (T(t')T(t)) = S(t-t'). The terms D^(x) 
are called drift coefficient (n = 1) and diffusion coefficient 
(n = 2) and reflect the deterministic and the stochas- 
tic part respectively, fixes the amplitude of the 
stochastic part and is referred to as dynamical noise. If 
depends on x it is called multiplicative noise; oth- 
erwise it is called additive noise. 

In recent years a parameter-free reconstruction of the 
coefficients and thus of the corresponding Langevin pro- 
cess has been achieved 0, H 0, H, El • It has been suc- 
cessfully demonstrated that traffic flow dynamics , the 
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chaotic dynamics of an electronic circuit 8, 9] or the hu- 
man heart beat rhythm can be reconstructed without 
need of any a priori models but just from measured time 
series and the estimated drift and diffusion coefficients. 
This estimation is based on the evaluation of the first 
(n = 1) and the second (n = 2) conditional moments: 

M^\x,r) = {[x(t + r)-x(t)] n )\ x{t)=x (2) 
from which the coefficients are derived according to: 



D {n) {x) = lim 



-M W (i,r) 



(3) 



For ideal time series with a sufficient temporal resolution 
the coefficients D^ n \x) can unambiguously be obtained 
from Eq. |(3} . For real data sets however the sampling 
frequency might be too low to resolve the dynamics prop- 
erly as it was pointed out in [Til IT^ . For small but finite 
r the conditional moments are better appr oximated by 
an Ito-Taylor series expansion (e.g. [h2l Il3t IT3|): 



mW(i,t) m tD (1 Hx)+0(t 2 ) 
M^ 2 \x,t) m tDW(x) + 0(t 2 ) 



(4) 
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Depending on the process it might be necessary to con- 
sider further higher order terms for estimating the co- 
efficients. To finally decide whether an obtained set of 
coefficients represents the real dynamics at least a consis- 
tency check between the statistical properties (moments, 
probability densities, etc.) of the reconstructed and of 
the original times series has to be performed (cf. (l5jl- 

Another important effect that complicates a proper es- 
timation of is the presence of measurement noise 
a((t) (with (((t)) = and (CWC(O) = S(t - t 1 )) which 
is superimposed on the data. Measurement noise cor- 
responds to a rather unavoidable experimental situation 



(e.g. 0,13113) and means that y{t) = x(t) + <r((t) is ex- 
amined rather than x(t). For instance, take the measure- 
ment of a turbulent velocity time series. The resolution 
is chosen in such a way that the largest fluctuations (on 
the largest time scales) are resolved. A certain amount 
of measurement noise might be negligible for these large 
scale fluctuations but can well be significant for the fluc- 
tuations on the smallest scales where the fluctuations are 
much smaller! More generally, the term 'measurement 
noise' refers to any superimposed uncorrelated noise that 
is present in some complex system; it might even be gen- 
erated by the complex system itself. 

To reconstruct the unknown dynamics x(t) from the 
accessible y(t) it is thus essential to quantify <x£(t) and its 
influence on the reconstruction of coefficients according 
to Eqs. © and which will be the central concern of 
this letter. 

in dum it has been shown that measurement noise 
results in an offset term, 7„, for the conditional moments 



M (1) (-1 M (1) (-1,t)/t 



M <1) (-1 ,x), M (1) (-1,x)/x 



M (")(y,T)^Af(")(x,T)+ 7 „, 

7i = ; 72 = 2cr 2 . 



(5) 
(6) 



Any non-zero offset causes a strong overestimation of 
the coefficients, D^ n \ because it leads to a divergence 
of M(")(y,r)/T in Eq. ©. In 8] it was therefore pro- 
posed to use the offset 72 to quantify measurement noise 
and to take the slope of the conditional moments (as a 
function of r) as an estimate of the coefficients. 

In this paper we will show that Eqs. (J5J and 10 are 
restricted to the special case of low measurement noise, 
that M^{y,r) must exhibit a r-independent part (i.e. 
71 7^ 0) and that the slope of the conditional moment, 
is no longer proportional to the corresponding co- 
efficient, . In Fig. ^the effect of measurement noise 
on M^ 1 is shown for two real world examples, in which 
a strong offset at r = causes a divergence of M* 1 ' /r. 
Finally, we propose an improved method to quantify mea- 
surement noise even for very large noise levels. To this 
end a general calculation of the conditional moments will 
be performed. Thus we can explain the measurement 
noise dependence of the second and the first conditional 
moment and propose an improved reconstruction of the 
underlying process. For an Ornstein-Uhlenbeck process 
the results are analytical, for non-Ornstein-Uhlenbeck 
processes the corresponding analysis can be performed 
numerically. 

Let us start with the calculation of the conditional 
moments from the accessible data y(t) spoilt by superim- 
posed measurement noise. Using the definition according 
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FIG. 1: Both plots show measured M^ 1 ' (squares) andM^'/r 
(circles) as a function of lag t. The left plot refers to the 
North Atlantic Oscillation (NAO) index )23lj. the right one 
to an increment time series of financial exchange rates 
Units are arbitrary for clarity of presentation. For both sys- 
tems a significant offset of the first conditional moment is 
observed causing a divergence of /t! 



to Eq. @ the following expressions can be derived: 

M (1) (y,r) = (y(t + r) - y(t)} \ y{t)=y=x{t)+rT(it) 

= tJ D^(x)f(x\y)dx + J(x-y)f(x\y)dx 
= m (1 \y,T)+^(y) , (7) 

Af (2) (y,r) = t [ \2{x - y)D {1 \x) + D^(x)} f{x\y)dx 



- y) 2 f(x\y)dx 



= mW 



m ( >(y,r) +72(2/) 



(8) 



The coefficients, D^ n \ are implicitly given by the condi- 
tional moments that exhibit a r-dependent as well as a 
r-independent part, denoted with m^ n '(y,T) and j n (y), 
respectively. According to Bayes ' theorem the unknown 
probability density f(x\y) is given by j f(y\x)p(x)dx wnere 
f(y\x) denotes nothing else than the distribution of mea- 
surement noise. Here we consider Gaussian distributed 
measurement noise with variance a 1 . The distribution of 
the process x(t), given by Eq. is denoted by p(x). 
For stationary processes the distribution is known to be 



p(x) 



A' 



D( 2 )(a 



exp 



DW(x') 
DW(x') 



dx 



(9) 



where J\f denotes a proper normalization factor cf. |20| . 
To extract the coefficients from the four equations of Eqs. 
(|7|) and (JHJ we assume for convenience (but not necessar- 
ily) that the coefficients can be modeled as polynomials. 
For instance, take the case of a multiplicative process 
with L> (1) = dux and £> (2) = d 20 + d 2 \x + d 22 x 2 . Then 
5 parameters (a, dn, d 2 o, d 2 i, d 22 ) have to be derived by 
minimizing the distance between the four measured func- 
tions, 7n(y), rh^ (y), and the solutions given by Eqs. Q 
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and ©, i.e. 



x(t) 



mini YyitVi) - 7i (y»)] 2 + [72(2/0 - l2{yi)f + 
'm^( yi ) - m^ivi)] 2 + W 2 Hy t ) - m^( yi )] 2 }. (10) 



For an Ornstein-Uhlenbeck process and for pure noise, 
Eqs. (0) and © can even be solved analytically. For the 
latter case (i.e. y(t) — <r£(i)) the moments are given by: 



M«(y,r) = -1 
M<V(y,T) = y 2 



(11) 



This means that for pure noise the moments as a function 
of t have vanishing slope but non-zero offset while for 
an ideal process according to Eq. Q the situation is 
reversed. Data from real processes will generally lead to 
M^™) values with non-zero offsets and non-zero slopes. 

Next we consider an Ornstein- Uhlenbeck process (given 
by £)W = —ax and = (3) to which measurement 

noise is added. In this case p(x) is a Gaussian distribution 
with zero mean and variance s 2 = f3/(2a). The offsets 



n(y) = -^2 'V 



72(2/) = 

and the to'"' values 



A 2 A 4 



• y 



(12) 



,(2) 



(y, T ) = T[-ay-oryi(y)] 

(y,r) = T[(3-2a(( l2 (y)-a 2 )+ yil (y))](13) 
can be derived exactly from Eqs. J7J and JHJ) (see 



for details). Note that A 



a has been used and 



that 72 approaches 2a 2 in the small er-limit in accordance 
with Eq. ©. 

From Eq. 113fl it is seen that the slopes of the mo- 
ments, m^/r, are affected by 7„. Thus simply taking 
the slope as an estimate of the coefficients - as suggested 
by Eq. © ~ is not appropriate in presence of larger mea- 
surement noise, even for rather simple cases such as the 
Ornstein-Uhlenbeck process. Estimates according to Eq. 
(|3*|) will be increasingly in error as "f n {y) dominates the 
conditional moments M( n \y,r) for large a. 

For illustration let us consider a numerical realization 
of an Ornstein-Uhlenbeck process with a = (3 = 1. Fig. 
121 a) shows the pure process (a = 0) and Fig. 121 b) shows 
the process with strong superimposed measurement noise 
(a = 1), corresponding to a negative signal-to-noise ra- 
tio of approximately S/N = 20 logio(s /a) = —3dB. 
Without measurement noise the coefficients are directly 
obtained either from the slopes of the conditional mo- 
ments or by using Eq. Q as shown in previous works. 
The reconstructed drift coefficient of Fig. [3 e) is found 
to be a = 1 ± 0.01 and analogously = j3 = 1 ± 0.01 
is well reconstructed (see Fig. 0f)). 
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FIG. 2: a) and b) show an excerpt of an Ornstein-Uhlenbeck 
process (r = 10~ 3 , a — 1 and (3 = 1) with a = in a) and 
a = 1 in b). In c) and d) the derived are shown, where 

c) refers to x = —1 and a — 0, and d) to y — —1 and <j = 1. 
In e) and f) the symbols represent the measured D^ n ' - fitted 
by the solid lines (a — 0) and m' n ' /r - fitted by the dashed 
lines (a — 1) according to Ea. U'A) . 



In presence of measurement noise the moments 
M^ n \y,T) are still linear functions of r but, in agree- 
ment with Eqs. 10) and JSJ), exhibit an additional offset 
term as can be seen in Fig. |2 d) . From the measured 
M( n \y,r) the terms m( n '(y,r) and 7„(y) are obtained 
as follows: 



(2) 



/r 
It 
7i 
72 



-(0.34 ± 0.02) -y 

(0.33 ±0.02) + (0.42 ± 0.01) ■ y 2 

(0.667 ± 0.001) -y 

(1.33 ± 0.02) + (0.445 ± 0.002) • y 2 (14) 



Using Eq. (12) we obtain the drift coefficient D^ x \x) — 
—a ■ x with a = 1.01 ± 0.02 in good agreement with the 
expected value of a = 1. 

To reconstruct the diffusion term = (3 the knowl- 
edge of 71 (y) and 72(2/) even at a single position y is 
sufficient when a is known. For instance, for y = — 1 the 
measured offsets are 71 = 0.65 and 72 = 1.74, leading to 
s = 0.73 and a = 0.99. With s = y / P/(2a) it follows 
that (3 = 1.01 ± 0.04. To improve the accuracy of the 
parameters a least squares algorithm is applied. 

Based on the foregoing discussion of an Ornstein- 
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Uhlcnbeck process, two important new results can al- 
ready be given. Firstly, we see that the estimation of the 
magnitude of measurement noise by the simple approach 
according to Eqs. (5) and (6) is misleading. For instance 
from the offset -j 2 {y = 0) = 2a 2 m 1.34 ± 0.02 a a 2 value 
of 0.67, which is about 67% of the real value, would be 
extracted. This underestimation has already been re- 
ported in |8( and can now be understood quantitatively. 
Secondly, if the small- r-estimate m^'/r is taken as an 
approximation of D^ 2 \ as it is commonly done, then an 
artificial quadratic diffusion term (see Fig. [21 f)) is ob- 
tained, masquerading as multiplicative noise or a bad 
temporal resolution |l2(. 

Finally we consider a general non-Ornstein-Uhlenbeck 
process where the coefficients, D^ n ' are implicitly given 
only by and -f n according to Eqs. |[7|-|3J). As an ex- 
ample, let us consider a process with multiplicative noise 
(£)( 2 ) — b + cx 2 ) and linear drift (D^ 1 ' = —ax) which is 
observed in various systems ranging from finance to tur- 
bulence (cf. ^JEIII^])- Here we take a = 1,6 = 0.1 and 
c = 0.5 and iterate numerically. The measurement noise 
amplitude is a ~ 0.25 which corresponds to a signal-to- 
noise ratio of S/N = 0. 
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FIG. 3: a) Symbols represent the measured offsets of the it- 
erated multiplicative process (D^ 1 ' = —x, Z)' 2 ' = 0.1 + 0.5a; 2 
and a = 0.25,). Solid lines represent the expected offsets ac- 
cording to Eq. and 0). b) Measured offsets and numerical 
solutions for the financial data set. In both plots circles refer 
to 71 and squares to 72 . 



Fig. a) shows the observed offsets, 7„, together with 
the (numerical) solutions according to Eqs. (J7J and © 
which agree well. Obviously the offsets are significantly 
different from those of an Ornstein-Uhlenbeck process 
(see Eq. (11211 ). Fig. [31 b) shows the observed and re- 
constructed offsets derived from the above presented fi- 
nancial data. Especially for 71, the multiplicative char- 
acter of the underlying process becomes evident. This 
result shows that multiplicative dynamical noise causes 
intermittent heavy-tailed volatility statistics in financial 
data as it was proposed, and this is not due to a spurious 
effect of measurement noise, as for the case of Fig. |2f). 

From the examples of Fig. 13 we see again that the un- 
derstanding of the influence of measurement noise on the 
conditional moments, i.e. on j n and m^ n ' /t, is the key to 
achieving a proper reconstruction of the underlying dy- 
namical process (including the contribution of dynamical 
noise and measurement noise) from pure data analysis. 
Naively applying the definition according to Eq. © will 
no longer be appropriate as soon as measurement noise 
is present. 

To conclude, we have shown (for numerical as well as 
real world data) that adding measurement noise to sig- 
nals generated from a Langevin process leads to a funda- 
mental modification of the data analysis via the condi- 
tional moments. A general equation describing this modi- 
fication has been presented and for the class of Ornstein- 
Uhlenbeck processes analytical results are given. This 
makes it possible to extract the strength of measurement 
noise, cr, the standard deviation of the underlying pro- 
cess, s, as well as the drift and diffusion coefficients, D^ l > 
and Z?( 2 ) , rather precisely even in presence of very strong 
measurement noise. It is noteworthy that the evaluation 
of the process' coefficients is solely based on analyzing the 
conditional moments, which are directly obtained from 
the time series without any need of pre-manipulating 
(e.g. filtering, modeling) the data. 
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